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ABSTRACT 

Motion  Picture  sets  are  traditionally  built  using 
decorated  modular  wall  components  called  “flats”.  The 
FlatWorld  project  (Pair  et  al.,  2003)  at  the  University  of 
Southern  California  Institute  for  Creative  Technologies 
merges  this  practice  with  immersive  technology  by 
creating  a  system  of  displays  coupled  with  physical  props 
which  can  be  scaled  to  simulate  entire  buildings  and 
streets.  A  single  room  prototype  FlatWorld  system  was 
developed  in  2001.  The  software  developed  for  this 
prototype  was  not  scalable  beyond  the  simulation  of  a 
single  room  environment.  In  2003,  the  FlatWorld 
Simulation  Control  Architecture  (FSCA)  was  developed 
to  support  multiple  digital  flats  in  arbitrary  configurations. 
The  FSCA  facilitates  digital  flat  training  scenarios  which 
can  be  scaled  from  the  simulation  of  a  single  room  up  to  a 
complete  city  block.  The  architecture’s  flexibility  allows  it 
to  easily  interface  with  a  variety  of  3D  graphics  engines 
and  display  devices. 


1.  INTRODUCTION 

The  FlatWorld  system  is  intended  to  be  a  scalable 
virtual  environment  which  can  simulate  the  experience  of 
being  inside  a  building  interior  or  outside  on  a  city  street. 
This  requirement  demands  a  software  solution  which  can 
manage  dozens  of  3D  graphics  displays  along  with 
surround  speakers,  tracking  sensors,  and  effects  devices. 


A  proof  of  concept  FlatWorld  system  (Figure  1)  was 
built  in  2001.  This  basic  system  consisted  of  two  displays 
simulating  a  room  interior  with  a  physical  window  and 
door  providing  views  onto  an  exterior  virtual 
environment.  Though  this  prototype  allowed  us  to  test  the 
core  functionality  of  the  FlatWorld  concept,  it  could  not 
support  a  configuration  that  varied  substantially  from  the 
simulation  of  a  one  room  interior. 


Figure  1:  The  FlatWorld  one  room  prototype  developed  in  2001 


To  move  beyond  our  prototyping  work,  we  required  a 
new  software  infrastructure  that  would  support  multiple 
system  configurations  of  arbitrary  scale  along  with  a  wide 
variety  of  immersive  audio,  3D  graphics,  and  user 
tracking  technologies.  To  meet  these  requirements  we 
designed  and  developed  the  FlatWorld  Simulation  Control 
Architecture  (FSCA). 


2.  RELATED  WORK 

A  number  of  software  libraries  have  been  developed 
to  address  the  unique  requirements  of  virtual  reality 
application  development.  CAVELib,  WorldToolkit, 
Avango,  and  VR  Juggler  are  widely  known  solutions. 
Most  of  these  toolkits  are  oriented  to  popular  immersive 
display  systems  such  as  the  immersive  workbench  and  the 
CAVE  (Cruz-Neira  et.  al,  1993).  Many  are  also 
distributed  as  commercial  products  requiring  licensing 
fees.  VR  Juggler  (Cruz-Neira  et.  al,  2002)  is  notable  in 
that  it  is  implemented  as  an  open  source  toolkit  and  can  be 
configured  to  work  with  different  types  of  immersive 
displays. 
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Though  VR  Juggler  satisfied  a  number  of  our 
requirements,  using  it  would  have  required  a  major 
reworking  of  our  existing  code  base.  We  were  satisfied 
with  many  of  the  software  design  decisions  used  in  the 
prototype  system,  and  we  had  no  need  implement  them 
again. 

3.  SOFTWARE  ARCHITECTURE 

The  FSCA  was  designed  to  enable  a  network-centric 
system  of  multiple  client  displays  driven  by  a  single 
controller  application.  Each  display  is  abstracted  as  a 
unique  entity  with  its  own  IP  address  and  camera  position 
within  the  virtual  world.  In  addition  to  graphics  display 
clients,  the  FSCA  also  supports  clients  which  manage 
tracking  devices,  audio  output,  and  effects  devices  (Figure 
2).  The  controller  application  broadcasts  user  triggered  or 
scripted  event  data  to  all  active  clients. 


Figure  3:  Three  digital  flats  depict  a  room  interior  and  a  view 
down  an  exterior  alley  corridor.  Each  display  is  driven  by  an 
FSCA  3D  graphics  client. 


Figure  2:  The  FSCA  controller  application  broadcasts  messages 
to  audio,  graphics,  tracking,  and  effects  clients. 

The  FSCA  is  optimized  to  run  on  Windows  XP  based 
systems.  The  FSCA’s  core  libraries  are  cross-platform 
making  a  Linux  or  Mac  OS  X  port  a  straight  forward  task. 
The  processing  load  is  distributed  by  running  the  client 
applications  on  multiple  networked  PC’s  which 
communicate  with  the  controller  application.  To  date  we 
have  developed  clients  for  3D  graphics  rendering  using 
game  engines,  OpenAL  based  immersive  audio 
processing,  and  sensory  effects  device  handling  using  the 
XI 0  home  automation  protocol.  Each  3D  graphics  client 
renders  its  scene  according  to  an  assigned  virtual  camera 
position.  By  having  the  capability  to  tie  cameras  to 
specific  digital  flats,  we  can  easily  transform  the  simulated 
environment.  For  example,  three  digital  flats  simulating  a 
room  interior  (Figure  3)  can  quickly  be  repositioned  and 
configured  to  display  an  outdoor  storefront  scene  (Figure 
4). 


Figure  4:  The  three  digital  flats  from  the  previous  figure  are 
reconfigured  to  depict  an  outdoor  storefront  scene  using  the 
FSCA’s  virtual  camera  assignment  feature. 

3.1  Network  Communication 

The  FSCA’s  networking  component  was  developed 
using  the  Adaptive  Communication  Environment  (ACE),  a 
cross  platform,  open  source  library  which  manages  a  wide 
array  of  concurrent  communication  tasks.  Above  ACE  we 
implemented  a  network  message  exchange  library  we  call 
NetMsg.  NetMsg  is  designed  such  that  there  is  a  single 
master  router  and  multiple  slave  listeners.  The  router  can 
broadcast  to  all  registered  listeners  or  to  a  single  listener. 

NetMsg  uses  a  multithreaded  architecture  to  handle 
reconnection  tasks,  message  buffering,  and  application 
notifications  via  C++  functors.  Delivery  time  can  be 
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assigned  to  each  message  for  events  that  are  time  critical. 
The  Network  Time  Protocol  (NTP)  is  used  to  synchronize 
PC  clocks.  Messages  are  programmed  as  C++  classes. 
Consequently,  messages  can  be  added  to  the  system  as 
long  as  the  new  message  ID’s  are  unique. 

3.2  Controller  Application  Graphic  User  Interface 

The  controller  application’s  graphic  user  interface 
(GUI)  (Figure  5)  was  built  using  the  open  source,  cross 
platform  wxWidgets  framework.  The  GUI  allows  a  user  to 
assign  virtual  camera  positions  to  display  clients,  select 
scene  databases,  trigger  animation  events,  enable  input 
devices,  and  activate  sensors.  The  3D  scenes  are 
controlled  by  a  sequence  of  event  action  states  defined  in 
a  custom  implemented  script  file. 


Figure  6:  An  OGRE  based  3D  graphics  display  client  depicts 
views  onto  a  city  street  through  a  physical  door  and  window. 


Figure  5:  Controller  application  graphic  user  interface  shows  a 
3D  scene  as  viewed  on  a  selected  digital  flat.  Buttons  on  the 
lower  right  trigger  scripted  events. 


4.  3D  GRAPHICS  RENDERING 

With  its  network-centric  infrastructure,  the  FSCA  is 
not  tied  to  a  specific  graphics  rendering  engine.  In  other 
words,  it  is  image  generator  agnostic.  An  FSCA 
application  can  be  authored  using  3D  game  technology 
such  as  NDL’s  Gamebryo  or  Epic’s  Unreal  engine. 
Alternatively,  the  FSCA  can  utilize  more  traditional  3D 
simulation  engines.  Consequently,  the  FSCA  can  easily 
integrate  components  of  legacy  simulation  and  training 
systems.  Our  first  FSCA  application  (Figure  6)  used 
OGRE  (Object  Oriented  Graphics  Rendering  Engine)  for 
3D  graphics  rendering.  OGRE  is  also  open  source  and 
supports  Windows,  Linux,  and  Mac  OS  X. 


5.  FUTURE  WORK 

In  the  coming  year,  we  plan  to  significantly  expand 
the  FSCA’s  core  capabilities  by  implementing  new  types 
of  clients  and  adding  support  for  new  protocols.  Also  we 
are  developing  FSCA  applications  to  test  and  validate  the 
architecture’s  functionality. 

5.1  New  Sensor  and  Device  Capabilities 

We  intend  to  develop  a  vision  sensor  client  which 
will  enable  FlatWorld  simulations  to  act  as  smart,  sentient 
environments  which  can  dynamically  react  and  adjust  to 
trainee  behaviors.  We  will  also  improve  the  FSCA’s 
ability  to  control  effects  devices  by  implementing  support 
for  the  DMX  show  control  protocol.  DMX  is  a  versatile 
and  robust  framework  which  overcomes  limitations  of  the 
X10  home  automation  protocol  which  we  are  currently 
using.  DMX  will  expand  FlatWorld’ s  multi-sensory 
effects  capability  by  allowing  us  to  more  easily  control 
devices  such  as  strobe  lights,  fog  machines,  and  air 
blowers. 

5.2  HMD  based  FSCA  application 

We  are  in  the  process  of  developing  our  first  FSCA 
application  targeted  to  primarily  use  head  mounted 
displays.  This  application  is  intended  as  an  experimental 
platform  for  treating  Iraq  war  veterans  with  post  traumatic 
stress  disorder. 
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5.3  FSCA  Large  Environment  Simulation 


CONCLUSIONS 


In  2005,  we  will  be  developing  a  prototype 
application  using  several  digital  flats  to  simulate  an 
expansive  scene  depicting  both  indoor  and  outdoor  spaces 
(Figure  7,  Figure  8,  and  Figure  9).  This  application  will  be 
appropriate  for  presence  patrol,  call  for  fire,  and  tactical 
questioning  training. 


The  FlatWorld  Simulation  Control  Architecture  is  a 
flexible  framework  for  supporting  a  broad  variety  of 
virtual  reality  software  and  hardware  configurations.  The 
system’s  versatility  allows  it  to  easily  build  upon  legacy 
simulation  systems.  Additionally,  with  the  FSCA  we  can 
create  training  applications  which  can  be  quickly 
reconfigured  as  demanded  by  the  rapidly  changing 
operational  environment.  Furthermore,  the  large  scale 
digital  flat  simulations  enabled  by  the  FSCA  will  give  the 
warfighter  the  ability  to  experience  an  environment  before 
physically  being  there. 
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Figure  7:  Depiction  of  a  large  indoor/outdoor  digital  flat 
simulation  driven  by  the  FSCA 


Figure  8:  Depiction  of  a  simulated  city  street  in  a  large  scale 
environment  FSCA  application 
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Figure  9:  3D  rendering  of  a  building  interior  within  the  larger 
simulated  environment  seen  in  the  previous  figures 
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